Remarks 

Claims 1, 6, 8, 10-11, 16, 21, 23-26, 28, 33, 35-38, and 40 have been 
amended, while claims 5, 15, 19, and 31 have been canceled without prejudice. 
Claim 41 is herein added. Applicant respectfully traverses the Office's rejections 
and, in view of the foregoing amendments and the following remarks, respectfully 
requests that the Office issue a Notice of Allowance. 

Allowable Subject Matter 

In the current Action, the Office deems claims 23-26 and 35-38 allowable. 
Specifically, the Office states that these claims "are objected to as being dependent 
upon a rejected base claim, but would be allowable if rewritten in independent 
form including all of the limitations of the base claim and any intervening claim." 
Office Action of 09/10/2007 . p. 18. 

Applicant thanks the Office for this indication. 

Expectation that Any Subsequent Action Maintain Non-Finality 

Applicant notes that new independent claim 41 has been amended to recite 
subject matter previously recited in now-canceled dependent claim 12. Applicant 
also notes that the Office agreed during the afore-mentioned interview that the 
reference (Pirolli) cited as teaching the recited "information fidelity" of claim 12 
fails to teach such an element. As such, the Office agreed that the outstanding § 
103(a) rejection of claim 12 fails to state a prima facie case of obviousness. 
Applicant again notes that new claim 41 corresponds to original dependent claim 
12. For at least this reason, Applicant respectfully submits that any subsequent 
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Office Action (other than a Notice of Allowance) should remain Non-Final. See 
37 CFR § 1.113, MPEP § 706.07(a). 

Drawing Objections 

In the current Action, the Office objects to Applicant's drawings. Applicant 
herein amends the specification and includes two replacement drawing sheets, thus 
obviating the grounds for the objections. Applicant therefore respectfully requests 
that the Office withdraw the objections. 

Claim Objections 

In the current Action, the Office objects to claims 8, 1 1, 23-26, and 35-38. 
Applicant has amended each of these claims, thus obviating the grounds for the 
objections. Applicant therefore respectfully requests that the Office withdraw the 
objections. 

§ 103 Rejections 

Claims 1-5, 7-10, 15-21, 28-33, and 40 stand rejected under 35 U.S.C. § 
103(a) as allegedly being unpatentable over U.S. Patent No. 6,232,974 to Horvitz 
et al. (hereinafter, "Horvitz") in view of Computation Modeling of Visual 
Attention, Itti et al. (hereinafter, "Itti"). 

Claims 6, 14, 27, and 39 stand rejected under 35 U.S.C. § 103(a) as 
allegedly being unpatentable over Horvitz in view of Itti in further view of U.S. 
Patent No. 6,353,824 to Boguraev et al. (hereinafter, "Boguraev"). 

Claims 1 1, 22, and 34 stand rejected under 35 U.S.C. § 103(a) as allegedly 
being unpatentable over Horvitz in view of Itti in view of Boguraev in further view 
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of in view of Perception-Based Image Transcoding for Universal Multimedia 
Access, Lee et al. (hereinafter, "Lee"). 

Claims 12 and 13 stand rejected under 35 U.S.C. § 103(a) as allegedly 
being unpatentable over Horvitz in view of Itti in view of U.S. Patent No. 
6,670,963 to Osberger (hereinafter, "Osberger") in further view of Exploring 
Browser Design Trade-Offs Using a Dynamic Model of Optimal Information 
Foraging, Pirolli (hereinafter, "Pirolli"). 

Applicant respectfully traverses the rejections. Nevertheless, Applicant has 
amended the independent claims in the manner discussed during the interview and 
during the follow-up phone calls with Examiner Daley for the sole purpose of 
expediting allowance and without conceding the propriety of the Office's 
rejections. 

The Claims 

Claim 1 recites a method comprising (added language underlined): 

• modeling an image with respect to multiple visual attentions to generate 
a respective set of attention objects (AOs) for each attention of the 
visual attentions; 

• analyzing the attention objects and corresponding attributes to optimize 
a rate of information gain as a function of information unit cost in terms 
of time associated with multiple image browsing modes , wherein the 
corresponding attributes for each attention object of the AOs comprise a 
minimal perceptible time for display of subject matter associated with 
the attention object ; 

• responsive to analyzing the attention objects, generating a browsing path 
to select ones of the attention objects ; and 
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• causing display of subject matter associated with each of the selected 
ones of the attention objects for at least a respective minimal perceptible 
time by panning the image according to the generated browsing path. 

In making out a rejection of this claim, the Office alleges that the 
combination of Horvitz and Itti renders this claim obvious. Applicant respectfully 
disagrees. Nevertheless, for the sole purpose of expediting allowance and without 
conceding the propriety of the Office's rejections, Applicant has amended this 
claim as discussed during the interview. Applicant respectfully submits that the 
cited references at least fail to disclose "causing display of subject matter 
associated with each of the selected ones of the attention objects for at least a 
respective minimal perceptible time by panning the image according to the 
generated browsing path." (emphasis added). During the afore-mentioned 
interview, the Office agreed. Applicant thanks the Office for this indication. 

For at least this reason, this claim stands allowable. 

Claims 2-4 and 6-14 depend from claim 1 and, as such, the remarks made 
above in regards to claim 1 apply equally to these claims. These claims are also 
allowable for their own recited features, which the references of record have not 
been shown to disclose, teach, or suggest. In addition, while some of these claims 
are rejected in view of other references, these other references similarly fail to 
disclose or suggest the language added to the base claim. 

Claim 16 recites a computer-readable medium comprising computer- 
program instructions executable by a processor for (added language underlined): 

• modeling an image with respect to multiple visual attentions to 
generate a respective set of attention objects (AOs) for each attention 
of the visual attentions, the AOs representing respective regions of 
the image; 
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• analyzing the attention objects and corresponding attributes to in 
view of a model for human browsing behavior, the model 
comprising fixation and shifting states, in the fixation state an 
interesting region of the regions is exploited for information, in the 
shifting state one region of the regions is replaced with another 
region of the regions as a function of image view manipulation 
operations such as scrolling or tabbing operations , and wherein the 
corresponding attributes for each attention object of the AOs 
comprise a minimal perceptible time for display of subject matter 
associated with the attention object ; 

• responsive to the analyzing, optimizing a rate of information gain in 
terms of space as a function of information unit cost in terms of time 
associated with the model for human browsing behavior to generate 
a browsing path to select ones of the attention objects; 

• causing display of subject matter associated with each of the selected 
ones of the attention objects for at least a respective minimal 
perceptible time by panning the image according to the generated 
browsing path . 

In making out a rejection of this claim, the Office alleges that the 
combination of Horvitz and Itti renders this claim obvious. Applicant respectfully 
disagrees. Nevertheless, for the sole purpose of expediting allowance and without 
conceding the propriety of the Office's rejections, Applicant has amended this 
claim as discussed during the interview. Applicant respectfully submits that the 
cited references at least fail to disclose "causing display of subject matter 
associated with each of the selected ones of the attention objects for at least a 
respective minimal perceptible time by panning the image according to the 
generated browsing path." (emphasis added). During the afore-mentioned 
interview, the Office agreed. Applicant thanks the Office for this indication. 

For at least this reason, this claim stands allowable. 
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Claims 17-18 and 20-27 depend from claim 16 and, as such, the remarks 
made above in regards to claim 16 apply equally to these claims. These claims are 
also allowable for their own recited features, which the references of record have 
not been shown to disclose, teach, or suggest. In addition, while some of these 
claims are rejected in view of other references, these other references similarly fail 
to disclose or suggest the language added to the base claim. 

Claim 28 recites a computing device comprising a processor coupled to a 
memory, the memory comprising computer-program instructions executable by the 
processor for (added language underlined): 

• modeling an image with respect to multiple visual attentions to 
generate a respective set of attention objects (AOs) for each attention 
of the visual attentions, the AOs representing respective regions of 
the image; 

• analyzing the attention objects and corresponding attributes to in 
view of a model for human browsing behavior, the model 
comprising fixation and shifting states, in the fixation state an 
interesting region of the regions is exploited for information, in the 
shifting state one region of the regions is replaced with another 
region of the regions as a function of image view manipulation 
operations such as scrolling or tabbing operations , and wherein the 
corresponding attributes for each attention object of the AOs 
comprise a minimal perceptible time for display of subject matter 
associated with the attention object ; 

• responsive to the analyzing, optimizing a rate of information gain in 
terms of space as a function of information unit cost in terms of time 
associated with the model for human browsing behavior to generate 
a browsing path to select ones of the attention objects ; and 

• causing display of subject matter associated with each of the selected 
ones of the attention objects for at least a respective minimal 
perceptible time by panning the image according to the generated 
browsing path . 
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In making out a rejection of this claim, the Office alleges that the 
combination of Horvitz and Itti renders this claim obvious. Applicant respectfully 
disagrees. Nevertheless, for the sole purpose of expediting allowance and without 
conceding the propriety of the Office's rejections, Applicant has amended this 
claim as discussed during the interview. Applicant respectfully submits that the 
cited references at least fail to disclose "causing display of subject matter 
associated with each of the selected ones of the attention objects for at least a 
respective minimal perceptible time by panning the image according to the 
generated browsing path." (emphasis added). During the afore-mentioned 
interview, the Office agreed. Applicant thanks the Office for this indication. 

For at least this reason, this claim stands allowable. 

Claims 29-30 and 32-39 depend from claim 28 and, as such, the remarks 
made above in regards to claim 28 apply equally to these claims. These claims are 
also allowable for their own recited features, which the references of record have 
not been shown to disclose, teach, or suggest. In addition, while some of these 
claims are rejected in view of other references, these other references similarly fail 
to disclose or suggest the language added to the base claim. 

Claim 40 recites a computing device comprising (added language 
underlined): 

• means for determining respective attention value and image fidelity 
of attention objects derived from an image, specified minimal 
amounts of size and time conditions under which to present the 
attention objects to a user, and user-browsing characteristics; 

• means for adapting the image for presentation on a small display 
device to present objectively interesting portion(s) of the image 
based in part on display constraints of the device, and attributes of 
the attention objects; 
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• means for calculating a navigation path through the image that has 
been adapted for presentation on the small display device, the 
navigation path identifying a browsing path from one objectively 
important portion of the image to at least one other objectively 
important portion of the image, respective image portions being 
represented by individual ones of the attention objects, the browsing 
path being generated as a function of desired human browsing 
behavior associated with time and space ; and 

• means for causing display of the image by panning the image from 
the one objectively important portion of the image to the at least one 
other objectively important portion of the image . 

In making out a rejection of this claim, the Office alleges that the 
combination of Horvitz and Itti renders this claim obvious. Applicant respectfully 
disagrees. Nevertheless, for the sole purpose of expediting allowance and without 
conceding the propriety of the Office's rejections, Applicant has amended this 
claim as discussed during the interview. Applicant respectfully submits that the 
cited references at least fail to disclose "means for causing display of the image by 
panning the image from the one objectively important portion of the image to the 
at least one other objectively important portion of the image." (emphasis added). 
During the afore-mentioned interview, the Office agreed. Applicant thanks the 
Office for this indication. 

For at least this reason, this claim stands allowable. 

Claim 41 has been added and recites a method comprising (emphasis 
added): 

• modeling an image with respect to multiple visual attentions to 
generate a respective set of attention objects (AOs) for each attention 
of the visual attentions; 
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• analyzing the attention objects and corresponding attributes to 
optimize a rate of information gain as a function of information unit 
cost in terms of time associated with multiple image browsing 
modes; and 

• responsive to analyzing the attention objects, generating a browsing 
path to select ones of the attention objects, the browsing path being a 
trade off of time for space or space for time; 

• wherein generating the browsing path further comprises creating the 
browsing path in view of a skimming image-browsing mode as 
follows: 

o splitting one or more large AOs of the AOs into smaller AOs; 

o combining AOs in close proximity to one another into one or 
more attention groups; 

o arranging the attention groups in decreasing order based on 
respective attention values; 

o for each attention group of the attention groups: 

■ selecting the attention group as a starting point; 

■ calculating a total browsing time and information 
fidelity for each path of all possible paths from the 
starting point; and 

■ if the total browsing time is greater than a browsing 
time threshold, discarding the path; 

o selecting a non-discarded path having a largest information 
fidelity as the browsing path, the browsing path connecting 
each of the attention groups. 

As described above, new claim 41 corresponds to original and now- 
canceled dependent claim 12. As such, new claim 41 comprises original claim 12. 
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In making out a rejection of original claim 12, the Office states that the 



combination of Horvitz, Itti, Osberger, and Pirolli render this claim obvious. 
Specifically, the Office makes the following finding of fact, among others: 

Pirolli discloses: 

■ a method wherein generating the browsing path further 
comprises, for each attention group of the group of the 
attention groups: 

■ selecting the attention group as a starting point; 

■ calculating a total browsing time and information fidelity for 
each path of all possible paths from the starting point; and 

■ if the total browsing time is greater than a browsing time 
threshold , discarding the path; 

■ selecting a non-discarded path having a largest information 
fidelity as the browsing path, the browsing path connecting 
each of the attention groups (page 33, right column, lines 26- 
31, page 36, left column, lines 6-16 and right column, Task 
Conditions section, i.e. Hard deadline). 

Applicant respectfully traverses the Office's rejection. Specifically, 
Applicant respectfully submits that the Office's finding of fact regarding Pirolli is 
incorrect. As such, Applicant respectfully submits that the Office fails, in the 
current Action, to state a prima facie case of obviousness. During the afore- 
mentioned interview, the Office agreed. 

Specifically, Applicant submits that the Office's finding of fact regarding 
Pirolli is incorrect for at least the following reasons: (1) Pirolli fails to teach or 
suggest "calculating a total browsing time and information fidelity for each path of 
all possible paths from the starting point", and (2) Pirolli fails to teach or suggest 
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"selecting a non-discarded path having a largest information fidelity as the 
browsing path, the browsing path connecting each of the attention groups". 

Generally, Pirolli describes tools that allow researchers to study human- 
computer interactions by modifying and testing different browser user interface 
designs. The abstract of Pirolli states the following: 
ABSTRACT 

B&BgtteK aad researchers of buman-cmiptsief jjateradoa 
need toofe that pmml the rapid explomtisa and 
ffi3Bage(t3£Tit of hypotheses about corr.p-lex urtwacJxeas of 
dessgtss, task ccia&i&fls, m& m& strategies. Dyftsmfc 
pffagtsBsaaBsig is iotsrajfec-e^ as a sseh a tool fef the mstym 
qf mfarmatsosj farapng tecktoJogje?. Th$ technique is 
Slustjassd is ih? contests of ths S^ertfja&er text 
cteering browser. Hypothetical wptwmMte ia tewser 
sp&ed and test dusienag are aaasakad in the context of 
varatsoas ia task dsadliass astd lift quality of the dooiaaect 
npesltory. A complex sad i^-tahafiv® set of teadssiiis 
me*®* tittm emn Ms simple sp<* of feetoss, Itostratiag 
the ",e r ensl uji 1 ' ofth. app 



Pirolli , abstract. 

With the general nature of Pirolli in mind, Applicant now reproduces the 
first cited portion of the text. Here, Pirolli explains a technique (entitled "dynamic 
programming") for testing the fitness of different browser user interfaces by 
analyzing the human-computer interaction on the different interfaces. Specifically, 
this passage states the following: 

Conceptually, it does this by 
searching tiuougib the (MErai possible paths of hrnim* 
computer mteracdOB, evaluating the coses aad v&fries of 
<M : exe&i pat&s, wd Mag the best paths In this maimer, 
&m wm Mi tftw fcesi-ess* feeSamasm of * user 



Pirolli , lines 26-31. 

Again, this portion of Pirolli, as well as Pirolli as a whole, relates to 
determining a fitness of a user interface for a browser. As such, Applicant 
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respectfully submits that Pirolli entirely fails to relate to "attention groups" within 
an image at all, as recited in Applicant's claims. Nevertheless, setting this aside, 
Pirolli' s discussion of determining a fitness for each of different user interfaces 
fails to teach or suggest the concept of "calculating... information fidelity". As 
such, Pirolli fails to relate to "calculating a total browsing time and information 
fidelity for each path of all possible paths from the starting point", as recited in 
Applicant's claim, (emphasis added). 

Additionally, Applicant respectfully submits that because Pirolli has not 
been shown to relate to "information fidelity", Pirolli has certainly not been shown 
to teach or suggest "selecting a... path having a largest information fidelity", 
(emphasis added). As such, the user-interface analysis that Pirolli describes 
similarly fails to teach or suggest "selecting a non-discarded path having a largest 
information fidelity as the browsing path, the browsing path connecting each of the 
attention groups", also as recited in Applicant's claim, (emphasis added). 

During the afore-mentioned interview, the Office agreed. In fact, during 
the interview the Office stated that Pirolli simply lacks the concept of "information 
fidelity". Applicant sincerely thanks the Office for these indications. 

For at least these reasons, Applicant respectfully submits that new claim 41 
stands allowable. 
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Conclusion 

For at least the foregoing reasons, claims 1-4, 6-14, 16-18, 20-30, and 32-41 
are in condition for allowance. Applicant respectfully requests reconsideration and 
withdrawal of the rejections and an early notice of allowance. If any issue 
remains unresolved that would prevent allowance of this case, Applicant 
respectfully requests the Office to contact the undersigned attorney to resolve 
the issue. 



Respectfully Submitted, 

Lee & Hayes, PLLC 

421 W. Riverside Avenue, Suite 500 

Spokane, WA 99201 



Dated: 2008/01/16 / Robert G. Hartman/ 

Robert G. Hartman 
Reg. No. 58,970 
(509) 324-9256 ext. 265 
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